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[ffttlMKDlGB] 

[H&K 2 ] WE*^i f H Lfc £ ft KaliE 
EttoS^SOffliarffi. 

S l Xtt 2 12$M WtflW&U 

0 c i:*1#ai:-rsiB*a 1 71S 3 o^-fftfr-EE* 
[IM&H5 3 J»E*7S>x*h*»»S-&Si:*fc:aiJ 

Jg/T^S^Hfficfc 5 gtfSS C t \C J; 0 fuE*7 '>*x 

[fl*«63 buE*7^x* H^illRTSftmifuE 
J: D tuE^-^^x * h fcHtrlMFeSS C t 

fcifttfct-aaw&K i Tis 5 ©^rftfr-wEttoa* 

[SPJMffl&IBBJH 

[00013 

CD X* tv W 7 t LX^ y§l<D XJl r >U 7*i&ffi Lit 

[00023 co^o^yMXttf/U^t^yT 

>^7x-Xi:LT(0?Sffl*^$nT^So 
[0 0033 

a»t> jp-r v >f > * ^ x - x %*h-t a it tbic t << n >w 
(Dmmzm i^tam* t <d^ k> t <o *Vr ? t a c u 

1 (Graphical User Interfac 
e) y^x-Xtfffl^&ftT^So f£#<DCU 
I •p«^ft^?77-*ffl^T^14*[Rl±?^T^ 



[0 0 0 43 

[00053 *»w»±Eo^fcfi»T**nfct>© 

[0 0 0 63 

M^»^-rafcto60¥is3 *m}<om#m i h 

few, l5E}g^¥a©ffiH*ttffi-rs*ffl¥a4:, tiu 

thije^t*^ x-7h <j)W&imtt%wmwm. t * 
[00073 w*9ii te*ntfx *^x^hoa^* 

tfT^x* h©«yi©itfTfcffi-3Tra&£Mf 3C tic 

MmviVtitm&mcmmxzz,, mmmza, trvx 

[0 0 0 83 ^5f?«2lC*ntf> *r^x^h%J»RL 

amufc*^x^h*its!«fc:jBaT»ts 0 

IB*« 3 tt> ^7~>*x ^ h *giJBJj-r S * t*rs?x * 
[000 93 3 lc <fc titf , ^-7i/*x * h ^iij^f 

atib. *7Vx7 b^mm^9m^mm\cmmx 
m*3S4ii, *7&x* bzum+ztzicam 

[oo i 03 n*94it<fen^ *7Vx*h*gijBrr 

^T^ci:lc<}:t)^7*S/*x^ h*«^nafcJ6, ^7 
x ^ H * iJ^f a «aa^ e^W^Kj^T^fTt? * a o 
M**5»4s *7^x^h%»»*-&Si:*k:JB^#S 
\c&K>*7i7x>? hltmiRLtcm. Jb^#a*BSJ: t) 

[0 0 1 13 il*«5tJ:tUf, *7"S?x*h*»tt« 
■ft* i: * tc ttjfi^aic j; 0 HcTV^ Y Lit 
»*¥a*HffiJ; 0 gflSS-ttS c (c<k 0 *7i?x 

7 h*»»*-tt*fc«>, w^m^^^fVx^ 



(3) 



HPII9-] 0 18 7 4 



[0012] nsJWtt, ^ys/*x^ h^ji^-r^K^ 

[0 0 1 3] 

^«t^0*^-r o **fi£^jcon>tfa-^ i «, ^ym 

fl^^Wcfc^;^, ^>2c7)tffg£&ttjf Sfcfe 
fcft©7VX7°W 4, *:7V-;> h 3 (CRISIS Lfc^ 

x h tcwf^s^^-r § i: ^ t t^a^s^^T-r 

[0 0 14] ?7l/7h3a> fflRifWHSgS^U 
•yr-J:9&9, ^>2<9ft^©27l^ h3<D«ffi-v© 
SMSt>"^SM^(D{iHRt;^>2 0^yU"y h 3<Dm 

yvv h 3T'^tB$n^^>2©tt^ac pu 5fcm*& 

$n§. r-f 7.7V-Y4«, 0IR14LCD (L i q u i 
d Crystal Display; (foUr -f X7°b 
-O £t)&t>, CPU 5, XB, ^'J 6fre>{&i&£ft 

[0 0 15] CPU5fC«^yu-y h3T'^HJ^nfc^ 
>2(7)WcM-r'l)1ffg*W$n, h 3fr£> 

[0 0 16] 0 2(C*^CO-HSSW©CPU<Dil^r 
D-y^0^fo CPU 5(4, r-3fr£>{ttii&£ 

n-5 ^ > 2 cot^s^-nt fafr ^ > 2 (ovmzmm? 

5^>««SMRffil 1, ^>«.^iig(5l lt'EIUc 
^ > 2 Wlftifc/S UTa— ^Jg/T Lfc ^ i7f+3 
v^K^EiCrs^xX^ + EISgPl 2. ->*i^7tl 
ISiS5 1 2 T'EISSft^xT,:^ RcjSCTJift 
Km {*7Vx>?Y) £)MKir3*7"^x^ r-gftlgPl 
3, ^7~>*x^ hJl«SI5l 3T«^nfc^y^x^ h 
tC^ LT ~>*x y Dfc kf^j. 7;bx7 x 
^ H^AHjStf^aT^x^x^ r-gfll 4 J;0«?n 

[0 0 17] C P U 5T'©^y^*x^ hCQlMttt, 

*mmt%mw&(r)ffif&m?k^o m?> (a) a, ->*x 



ivmi&m. H3 (b) «, ->*ix 

To 

[0 0 18] ^*x7>^-V^lSS2 1 li->'iXf+i 

tfWMz ft 5 s -eic^f $ nzmmttfew £ ftfc«3S«i 

lflciD«?n, -I®i/iXf+«(c, fill 
£te#i£ftT^3o 0 3 (A) (Cti2ffl^<D^«)Jaa 

( 1 ) , ( 2 ) t &'iJi«S t<D3 fiSMStf Hti2 ft 

[0 0 19] ~>*xX?--f 3f g a«, ^>2©»^tcJ;0 
EISI-^^^LTfcO. CCT'te, IK (1) , 

(2) , mm (1) , (2) , 0My\- (1) , (2) 
*^?ftT^5 0 (1) tt, #7"i/*x*r- (7^3 

x 7° 4 ffifr e. «m « mri ic & o r << a 

f«S>iXft*SLT^4. ilJR (2) 14, ^ 

y U 7 h 3 * ^ L T r Y 7s -f Vy 4 Sic * y 2 ? 

■stcwigv. jj^yoimmtntsi/xT.^MttTr^T 

» (1) , (2) ©i/xXf L + t<fct)7'rn> 
[0 0 2 0] «»f (l) ti, 7-ra>**ft»6fe 

icmwzmfttts mm (2) it, 7-fay^t^&c 

«WI5i?z7f + *Sl/T^8, «J»f (1) , (2) {C 

(1) 14, ^>25:^7U7 r- 3^LTr^X7' > W 

y2*?7l»y h3^^LTr-<X7 0 U"l'4S<fct)gt-r 
Uft*SLTfc>*. (Ox (2)©->'iXf 

[0 0 2 1 ] tt»J»S^Sbtett, ->'iXftfSat' 

? ftfc y x x + js u r^ffi-r 5 KffiSb^ 

$ftT^5 0 9M*J»*i:LTtt, 74uy\c&itm% 
7^ny<D&*miktZ>%.^ K<D7-(3y<m 

vfc$W>fc«J»r8!&Aft3£^ 7^3y\c1-¥lcWm 

mzxn, 7^3^*^s-r*a^*^sssftTo^s, 

[0 0 2 2] SiiMSctC«, ^OS;x^ + *qgK* 

ftss-pfcUff-r^ftiaj&^sftTc^. sw 

ct»3£*ft*«affli:LTttx 7^n><D&»j£pJti£: 

?%7-(^y®mwm, ^->"xxft^i:^Bt 

S^ftT^So 

[0023] *fc, #-r>?^aa2 2«> ^a?ft5 

■T5fttt0^igi«^xX^ + Sge«fe»)«lfiR?ft, 
Sf§ ! SSacO^0^^-ro 14 (A) 14, -^iXf 



(4) 



9- 10 18 7 4 



**-+«2 3©*toKBk 0 4 (b) a, Hi§»sic 

CO 0 2 4] KKS?xX*-+a2 3fct, fiXffif^ 
^3^2 l©^7x7^+«Sa£fa£2ftfclft 
(1) , (2) , «lff (1) , (2) , 
(1) , (2) A^C^75Jfil^SA^f^7^y 

f U -b -y h 7? 7"g , ^i^t ©EiBrfM^jLfcfrS 

x? h*3ti-»w^i<' i <fc9#t$2ft 

So 

[0 0 2 5] H8&7-77f «, ^tC^xX^ + ^HtTRl 

HftWitt-feyh (" i *) 2ft, -^ixfttfiw 
piui**atty-b-y h Co') 2fts<, f&fr-s, * 

tUfr^ft*'^*^ i^ft^UT^S, U-b 
£}c«-fe>yh (' 1 ') 2ft, i/*xX^+©Ei8*WT 

2ftT^§a*icau-b7 h Co") 2ft5 0 

[0 0 2 6] £fc, BMWi5!j77^hJ4, ->*iXft© 
BMfflatffrto ft, fiWt #18182 tilt 1 * K it -fe 
(T) 2ft, J/'iXf+miSoMi: 

t, antttu-tv h co~) an-tv*, us? 

ftfc*7~>'x^h ^(^tfffi-T^toO^V 

[0027] siiMf aS2 4 a, 5>x**+JWWi 
5 j , &<mm&m^<o^M*^t^im7 : y>? v. a 

9«j£2ft5o W«l79^J«, S>x**-*j!><Kil*ft 
TSil4^tffTt>tt3£*«\ -b-vh C l ") 2ft 
T, ffia#flfT2ftT(/^ U-b-yb C 0 
*) 2ft3o 

[0028] ^?b77^ktt, mii^a^c© 

a«DHtf^T^TOc^o/c:i:*fc, ■b-j'h C l 

' ) 2ft, aura y -b* Co') fc«#2ft 
x^f-+^BK«ft*^^*fc'fey H2fts 0 aa-rs 

£?lc, C(OWS)77^k^-b-j' h2ftfct£{c«i/ 
7f A£f*#flMI:2ftT, ->7xA##-£WcK§£> 

[0029] s-r, &Ki (i) teato^Tittwrso 

0 5 fc*«WO-**Wl08*ffia ( 1 ) ©MfHMWH 
fc^f. $»#ia (1) T*«, Aftllcjgft (l) ->*iX 
f- + *ffi*LJ:5i:K#So CCT>, SS? ( 1 ) 7x7, 

^•vtt, ®5 (a) icT^-r^oicf^^y'u^ 4icB.7Ts 

2ftfc7f'3>'3 l«SO*7'^x^ Hc£7U-y h 3* 
1\LT$ -y^LTt^V 2 fcHffifr&SStf 



5, ^>2^fg^LT^-5fiH«, Hffi±fc^> 
A-7;l/3 2i:bTg^2ft5o ^>*-7;l/3 2tt, 

[0 0 3 0] C©jM3R (1) ^xX^+fcEiSLfci^ 
fc > 2 Wag 2 2 © 5? x 7 f- -V J*ft£ d 
(1) lct2£2ft, *Biyi7firfl(Se^xW 

+j*fewau8 2 \ <D®wimm d) ©^xx^-visia 

©2fS§£/Tt' 2 ' tf»££ftSo Sfc, 
MfnF9«2 l©^&S«ib#£«S2ft, 0 5 
(B) tc^t<fc'5fJ:K#?c!lSi:LT7-r=i>3 lfc»3 
3*Wtf £ft£o <:©i:t©*Iil£iac i: LT^©^* 
+ (1) '>iXf + «Jfl2i 

[003 1] 3Mc, #^7h (1) 5>x*f- + tfE»2 
7-f 3>3 l»C#^P)ft^3 7 
-Yn>3 1 ©&»j;W¥±2ft5o COt*, $»$aa 
(1) THi, 05 (C) fc^-fj;? tc7^3> 3 1 ©& 
IStffftt Ltn t £ &c jc©7-i' a VffiSlc 7^ 3 > 3 1 * 
£fi&S«T'S^2-e3o Celt*, -^B#MrtH:0 5 (D) 
9 fcfflf'O 2 tc i 07cOffiB©«[»O7-i'3 
>3 l ' *»ttbfc4:*»ctt, 7c©7-<3>'ffiHt7t© 
7-<3>3 1 ' *Ht;*«li:LT*^*^ 7^37© 

( 1 ) 5>xXf- + K»B#fc*ljg«iac t LT^^V-*'! 

mmzti, ^o^xxf-+T»*sas? (1) 7xx^-v 

*^^v-*^ft5fltjtcSISS2ftfci;tlcti, 
3lgbcD7-Y=i>'3 1 ©SSSWffbnS. 

[0032] #fc, ^aMa (2) tco«,^Kwrs. 
B6»c*»wo-^ifl60<jo»»«ia (2) (omimmm 

^Tjkto Ml (2) T'tt, 0 6 (A) fC^-r«fe^lC 
(2) t«fct)^>2«:r-rX7'U'('BSfc:»«i2-a-r 
t«7-Y3>3 1 ©iiH?rHtf<fe^lc 
<ii:{c<fc!3 7-Y3>3 i ©JISWfTtoft, 06 
(B) {C^f c k9tc31^2ftfc7'r=i>3 Uc»3 3^ 
mztiz* cot*, ^-rv-^»2ft, -^B#F^ 
rttc^cr)->*x7g L +T$>§^-i'>h (l) #E!i2ftft 
Hf, 06 (C) «fc5fc7>f3>3 ia%<E>tfc«P 

o*»6#-r>h (i) T-}i7^2ftfcfiBP nc7-r=i7 

3 1 *^»2ft5o 

[00 3 3] S/c, -^H*^giaLTtH©^xX^ 

tttft>tltc&3 3WS*2ftSo ^-Y7h (1) <?»*x 
7^ L + lCcJ;D7-t'3>'3 1 A^»2ft^:Ji-&{ctt, ^fc 
W>h (2) ©^i^ftOBi*fifllt«, Cl 
T\ (2) cO~7xX^-v^SS2ftSi:0 6 

(D) Ic^2ft5<fc'5lc7-Y373 lfc«»S»*i:LT 

f^^e>ftfc^*^S2ft?> e 

[0 0 3 4] ^IC, iiJ^aiCO^TltJB^f§ 0 0 7 (C 



(5) 



whfps- i oi874 



ffiaii. «JK (i) x (2) ©2ooi?i7fttit) 
*H<&ftTl^*. 0 7 (A) \z^,f^^,\L^yz 
Z7J=i>3 \ ±Z45frZtE\Cli%9]t>-&Z>Wm (l) ~>* 
x** + fc<J:»)Bl7 (B) t*-r<fc9ti!W»a*i:LT 

■frSttK (2) i/'i^f^iittilf, 07 (C) ic 

[0 0 3 5] -3&9n>WML&^rtlcH7 
(D) \C7Fi?7'fa>3 \<DmfflZmts&olc^y2* 
&®1<*k%WR (2) <Di?xX?*Z'n5ctlc£*), 

£, -J09IIIMBlLTfc«»r (2) (2) <DV 

[0 0 3 6] JJt±<D<fc3k^ £SMQS (1) , (2), 
*2 K #7>'* < taS2 2, BIi^iXf-ta2 3, 

ia»s«a«2 4icfleoTCPu 5jWft^«n*cfc 

»O:0*fT«nSo H8te*»W0-3ttfc09©CPU© 

[0037] CPU514, tt^M^tttS-f , ^t'J 

2 3, MJi«aaWS«2 4*mm\t?Z> Uf7 7"Sl 
-Do CPU5it ^>tt!BB«ttP 1 l J: 0 

0^>2 0«ttllHB*JR»r* (Xr-y 7°S 1-2) „ 
[0 0 3 8]at, CPU5tt, ^xX^ + &|Rg|5l 2 
icj; 0Xr>y 7*S 1 -2T»W»Ufc^>2©flHi^6a 

W.~?% (Xr-y/S 1-3~S1-8) 0 CPU5«, 
S>x*f- + *EKf.5 4u ->'xXf tH^flg2 1?: 

a£fl;&?-S£ttfc»>*xX:7 L +82 3<DrtS£H#r 
f5 (X-r-y7°S 1 -9, S 1 - 1 0) e 
[0 0 3 9] £tc, CPU 5^ SEttTBMUfcS'x;* 

7+tc/si;T->*xxf--\'»taS2 1 fcSjesnfcK 

«»S*S^U Xr-y 7°S 1-21C55S (Xf--y7°S 
1 - 1 3) o EI$£ftfc^xXf L + #?*ftL&^ 

jf^R t; x x 7 + ©his tc l «^ ic a: , mm 

mtmMLtj:^7.ryfS l -2B3tK Sl>VxX7 
^OWmZtlMt* (Xx-yT'S 1 — 1 K S 1 - 1 
2) . 

[0 0 4 0] C P U 5 ±fEX7 -y 7°S 1 - 2 ~ S 1 

-l 3*^0jsu f^7"i/-f 4 fcg^^nfcz-m 



To 

[0 0 4 1] Xr-yT'S 1 -3<D)13R (1) ->"iXf+ 
Z<DM%i7=7yf*Wm-% (X7-y7°S 2- 1) . C 

So sa677^f tfb-y hjnr^ntf, *>*xx 

* + *B»7?*5tt|gT&3;fci&, &fcKK5>x**- + 
«2 3©'Jb-y h75^g£8&nT3 (Xr-y7S2- 
2) o 'Jb-y F777'g^7 HSftT^tltf* KK* 

1\ ^y2#fV*7V^4fc»MUT^*fr5*\ r 
&fc*K ^>2B7U-> h3tcSMLTI/^^SA^ 

WBfsns (XT7/S2-3) o ^>2^xyu 

fcg«x-**$fc»tf«fcS£-r3 (X7-y7°S2-4 

-S2-6) o u±ic&9mm{tm*7?%o 

[0 0 4 2] &teflH§Lfc£«7 9 -**£«y*htei6 
iHTtS (XT7^S 2-7) o <1CT\ X7'y7°S 2 - 

2T-yb-y h7^^*g^yb-y h*nrc^5«^tctt, 

Xf7 7°S 2 -7T*l«ttTffi«T r -*#MiDSftSo C 

c-c, Man* nfc«f -?*wsyx h©jfe»*«^ 

(XT77*S2-8) o Xf7 7 p S2 
- 8 -V* y 2 CDjg^fuBtf 5fcIMf<D 1 * tt s S 7£8 

jr (i) *sis??nfe««"T»ttft^feai>, ©a«i7f 
t*»ca:g!jR (i) teawf*»%?jBW*fci6 

(7.7--y7 <> S 2-9) 0 

[0 0 4 3] Xr-y7°S 2 -9T'^>2(?)J$ffi7b^1Sy 

xhojfeK«a6fi«fc4^i:*fctt, a«? (i) *«as? 

^gfcb-y hLT«!S**rrr5 (X-r-y 7°S 2- 1 
4) o Xr 7 y S 2 - 9 T"<> 2 OSl*'i«U 

( 1 ) a^Stt«BjflB1±#*S;fc«>, *fcW? ( 1 ) 
CD-7'x X^-v -T*fe?>^> 2 #r -c X7*W 4 (C&fti L 
Tt^**»S^««PJW«n* (Xr-y 7°S 2 - 1 0) o 

[0 0 4 4] Xr-y7°S 2 - 1 0 T'^> 2 #f -r X7°U 
f 4K»«lUTl^ftttntf, S«? (1) «JWct 
WKT*ftt^«>, ^a^*?7-TSo Sfc, Xr-y 7S 
2-1 0T?^>2^7VX7Vf 4 fc»»LT0^41tf, 

»R (i) *qHR*nfcfcW»fTf*, 3fclc<>2#qBS 
S 2- 1 1) o 



(6) 



!f#HS¥9- 1 0 18 7 4 



[0045] 7f7^s2-i ie>mmvMMks ^>2 

(l) lc<fcD3I^nfc7^3y<bLT»7x 

yimv* i ics^$tifc7^ny^i5Siji-si* 

«%i9^-r« (Xf'^S 2-12, S 2 - 1 3) . $ 
fc, XT7/S2-1 l©85K©*S*x ^y2tfS*L 

^^n^ttEM"?**^** JIK (1) ©'J-b-y 
h79^g*-fey h bTJaa**Sf7f * (Xr-y7S2 
-12, S 2- 1 4) o 

[0 0 4 6] WJWC*?>*R (1) ©S/xX^ + affiBK 
$n?) 0 01 0t*S6WO-^B»W<MW? (2) £BK 
■TSfc46©Ma©i!l^7D-9 L + — h^rSTo Xt^T" 
S 1 -4©iHR (2) 'AXfttHWSftftOilli 

fcrTS (X7-v7°S3-l) o CCT% W»75^f* 1 

f # b -y h £ nr i/ ->ntf , ^"ixft ^^jst* * zim 

H&Zfcib. 3WcB*5>x**+a2 3©U-b-y b7^ 
yg*tt»rr* (Xr-y 7°S 3-2) o 'Jt^77? 

A^^u^^LTt/^fcto, 3;f, ^>2tfrfX7 0 

U 7 hafcattLT^SfrSfrtfWBrStt* (Xx-y7 
S3-2) . ^v2#7^X7V^4te»ttLT^51§ 

S. ^>2^7^x7W4fc^ttLTt,^^^ 
£lc&, U-b-y h772*g*U-b-y hU -<>2©-« 

WcKTrS (Xr-y 7°S 3-4-S 3-6) . W±t 

[0047] %uc%mLrcmMT-9*&mvx hta 

ftrTS (Xr-y 7°S 3-7) . CCT% X7'y7°S3- 
2T*Ub-y H77^g* , U-t , y h?ntl/^^lcS, 

X7b^ 4 fc»«LT^*jWgfrtfffl 
BrSnS (Xr-y 7°S 3-8) 0 
[0 0 4 8] X7'y7S 3-8T^2^^7'W 
4lCg84bT^§ii£K^ jift (2) ©^xX7 + T- 

1CJ -fe-y F77^"g*-fe^ h LT$&S£*l7-f 5 (Xf 
•y7S3-9) o X7-y7S 3-8T?^y2#7 

^x7l^4tcgMLT^&^i§^;:&, »ft (2) © 
-7xX7 + T-S§RjHl1t^fe2.^46, ^-©fc*^>2^ 
}^ L, X I ^3 ffi^^ y 2 ©flB^ft L T ^ 5 ffilf U 
xhfciiiiirrs (xr-y 7S 3-7) . 
[0 0 4 9] Xr-y7S Z-TV&M»7.Y\cW&f— 

9i^mc\tma^ti^>iL, at* sains nfcffi«7*-* 



A^tRU X H©JtSStf«*^§*»^!pJ»r«n5 (X7-y 7 
S 3- 1 0) o Xr-y7S 3- 1 0T^>2©f§^ilB 
jWHMHW>fc*fctt* *1£XR (2) tfSJRSttfctf 
flgT?ttfc^;fca&, «&iS£7-f3o $fc Xr-y 7°S 3 
- 1 07«r-^^7 H cIS^1«THi^^«k*lcti^ 
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(54) METHOD FOR CONTROLLING DISPLAY 



(57)Abstract: 

PROBLEM TO BE SOLVED: To provide a display controlling method enabling a 
user to intuitively and easily recognize the process of operation in respect to a 
computer display controlling method using a pen as an input device. 
SOLUTION: The coordinates of a pen 2 on a screen and contact/non-contact 
between the pen 2 and a tablet 3 are detected, and in the case of moving an icon, 
a required icon is selected by the pen 2 and then the pen 2 is separated from the 
tablet 3, so that the selected icon is displayed as if the icon is shadowed and 
lifted up by the pen 2. In the case of deleting the icon, a cutting line is displayed 
on the icon by executing operation for cutting the icon by the pen 2, so that 
operation can be intuitively grasped. 
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CLAIMS 



[Claim(s)] 

[Claim 1] The display-control approach which carries out [ having a detection 
means detect the location of said directions means, and a display-control means 
control the display of said object according to a motion of said directions means 
detected by said detection means, in the information processor which processes 
this object by directing directly the object by which a screen display was carried 
out with a directions means, and making a sequential change according to 
advance of processing of the display of said object of an object, and ] as the 
description. 

[Claim 2] The display-control approach according to claim 1 characterized by 
giving a shadow to said object when said object is chosen. 
[Claim 3] The display-control approach according to claim 1 or 2 characterized 
by displaying a cutting plane line on said object when deleting said object. 
[Claim 4] Claim 1 characterized by performing actuation which cuts said object 
with said directions means when deleting said object thru/or the display-control 
approach of three given in any 1 term. 



[Claim 5] Claim 1 characterized by attaching a shadow to said object and moving 
said object according to migration of said directions means by making said 
directions means estrange from a screen after choosing said object with said 
directions means, when moving said object thru/or the display-control approach 
of four given in any 1 term. 

[Claim 6] The actuation which chooses said object is the display-control 
approach of five claim 1 characterized by being the actuation which surrounds 
said object with said directions means thru/or given in any 1 term. 



DETAILED DESCRIPTION 

[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] This invention relates to the display-control approach, 
and relates to the display-control approach of the computer using a pen as an 



input device especially. In recent years, the devices which equipped the input 

device of a pen mold as an input device of a personal digital assistant device are 

increasing in number. The input device of this of a pen mold is because the input 

approach has agreed in human being's feeling unlike a keyboard. 

[0002] Such a pen blocking force device can be effectively used also as a 

pointing device, and the activity as an input interface will be expected from now 

on. 

[0003] 

[Description of the Prior Art] in order to realize an intelligible interface intuitively 
in a computer etc., graphic forms, such as an icon, are used and the exchange 
with a user is performed -- being the so-called - GUI (Graphical User Interface) 
-- the interface is used. In the conventional GUI, operability was raised using 
various metaphors. 
[0004] 

[Problem(s) to be Solved by the Invention] However, in the conventional GUI, 
since the process of the actuation which the user is performing was not reflected 
in the display, there was a trouble of which phase of processing has actuation 
being unable to recognize intuitively. 

[0005] This invention was made in view of the above-mentioned point, and aims 
at offering the display-control approach that a user tends to recognize the 



process of actuation intuitively. 
[0006] 

[Means for Solving the Problem] It carries out that claim 1 of this invention has a 
detection means detect the location of said directions means, and a 
display-control means control the display of said object according to a motion of 
said directions means detected by said detection means, in the information 
processor which processes this object by directing directly the object by which a 
screen display was carried out with a directions means, and makes a sequential 
change according to advance of processing of the display of said object of an 
object as the description. 

[0007] Since the sequential grasp of the processing situation of an object can be 
carried out by making a sequential change according to advance of processing 
of the display of an object of an object according to claim 1, a processing 
situation can be grasped intuitively. Claim 2 is characterized by giving a shadow 
to an object, when an object is chosen. 

[0008] According to claim 2, by attaching a shadow to the object chosen when 
an object was chosen, the selected object loses touch with a screen, is 
displayed, and can grasp the selected object intuitively. Claim 3 is characterized 
by displaying a cutting plane line on an object, when deleting an object. 
[0009] Since according to claim 3 it is displayed as if the object was cut on the 



screen by displaying a cutting plane line on an object, when deleting an object, 
the processing which deletes an object can be grasped intuitively. Claim 4 is 
characterized by performing actuation which cuts an object with a directions 
means, when deleting an object. 

[0010] Since according to claim 4 an object is deleted by performing actuation 
which cuts an object with a directions means when deleting an object, 
processing which deletes an object can be performed in intuitive actuation. 
When moving an object, after claim 5 chooses an object with a directions means, 
by making a directions means estrange from a screen, it attaches a shadow to 
an object and is characterized by moving said object according to migration of 
said directions means. 

[0011] Since according to claim 5 a shadow is attached to an object and said 
object is moved according to migration of a directions means by making a 
directions means estrange from a screen after choosing an object with a 
directions means, when moving an object, a display as if it raises an object with 
a directions means and was making it move is performed, therefore migration of 
an object can be grasped intuitively. 

[0012] Claim 6 is characterized by the actuation which chooses an object being 
actuation which surrounds an object with a directions means. Since an object 
can be chosen with feeling which surrounds the object chosen by considering 



actuation which chooses an object as the actuation which surrounds an object 
with a directions means with a circle according to claim 6, an object can be 
chosen intuitively. 
[0013] 

[Embodiment of the Invention] The block block diagram of one example of this 
invention is shown in drawing 1 . The computer 1 of this example is the system 
which has the input device of a pen mold. With the display 4 for displaying a 
tablet 3, processing information, an icon, etc. for detecting the condition of the 
pen 2 for a user to direct processing and a pen 2, and a tablet 3 While giving the 
image effectiveness to the object displayed on the display 4 according to the 
condition of the recognized pen 2, it consists of memory 6 grades holding CPU5 
which performs various processings, processed data, etc. 
[0014] A tablet 3 consists for example, of an electromagnetic-induction mold 
tablet, and is considered as the configuration which can detect the contact and 
the non-contact one to the front face of the location at the time of non-contact [ to 
the front face of the tablet 3 at the tip of a pen 2 / the contact and non-contact ], 
and the tablet 3 of a pen 2. The condition of the pen 2 detected by the tablet 3 is 
supplied to CPU5. A display 4 consists of LCD (Liquid Crystal Display; liquid 
crystal display), and displays the image according to the indicative data supplied 
from CPU5 or memory 6. 



[0015] the display control of the object which performs processing and displays 
migration, deletion, etc. of an object on a display 4 so that it may mention later 
according to the information on the condition of the pen 2 which the information 
about the condition of the pen 2 detected by the tablet 3 was supplied to CPUS, 
and was supplied from the tablet 3 - it carries out. 

[0016] The functional block diagram of CPU of one example of this invention is 
shown in drawing 2 . CPUS The condition of the pen 2 supplied from a tablet 3 
The condition of a pen 2 from the shown information It responds to the gesture 
command recognized in the gesture recognition section 12 which recognizes the 
gesture command which the user directed according to the condition of the pen 
2 recognized in the pen condition recognition section 1 1 and the pen condition 
recognition section 11 to recognize, and the gesture recognition section 12. For 
actuation (object) It consists of the visual effect sections 14 to which the visual 
effect according to a gesture command is applied to the object chosen in the 
object selection section 13 to choose and the object selection section 13. 
[0017] Actuation of the object in CPUS is managed by the gesture actuation 
management table 21 set as memory 6. The block diagram of the management 
table which manages gesture actuation of one example of this invention to 
drawing 3 is shown. The block diagram of the pointer management table 22 
where drawing 3 (A) manages the block diagram of the gesture actuation 



management table 21, and drawing 3 (B) manages gesture actuation is shown. 
[0018] The gesture actuation management table 21 consists of progress 
processing items c in which the processing which will be performed by the time 
the image effectiveness item b in which the gesture item a in which gesture 
actuation was stored, and the image effectiveness according to gesture were 
stored, and the following gesture are recognized was stored, and is stored in 
order of processing for every gesture of a series of. Three kinds of processings 
with two kinds of migration processings (1), (2), and deletion are registered into 
drawing 3 (A). 

[0019] The gesture item a shows the actuation which should be recognized by 
motion of a pen 2, and selection (1), (2), cutting (1), (2), the point (1), and (2) are 
shown here. Selection (1) shows the gesture which chooses an icon by actuation 
which makes the pen 2 in contact with an object (icon) estrange from the 4th 
page of a display through a tablet 3. Moreover, selection (2) is in the condition of 
having contacted the pen 2 on the 4th page of a display through the tablet 3, and 
shows the gesture surrounding the perimeter of an icon. An icon can be chosen 
by the gesture of selection (1) and (2). 

[0020] Moreover, cutting (1) shows the gesture to which cutting (2) crosses an 
icon for the actuation which crosses an icon on the left from the right on the right 
from the left. An icon can cut up and down by cutting (1) and (2). The point (1) 



shows the actuation to which a pen 2 is contacted on the 4th page of a display 
through a tablet 3. The point 2 shows the actuation which detaches a pen 2 from 
the 4th page of a display through a tablet 3. The pointing of an icon can be 
performed by the gesture of the point (1) and (2). 

[0021] The image effectiveness carried out according to the gesture recognized 
by the gesture item a is set to the image effectiveness item b. As image 
effectiveness, a cutting plane line is put into a cross joint at the display which 
attaches a shadow to an icon, the display which eliminates the shadow of an 
icon, the display which makes the original icon location a broken line, the display 
which continuous-line-izes an icon, the display which puts a cutting plane line 
into an icon aslant, and an icon, and the display which eliminates an icon is set 
up. 

[0022] The processing which should be performed by the time the following 
gesture is recognized is set to the progress processing c. As processing set as 
the progress processing c, the icon migration processing which enables 
migration of an icon, and timer starting processing in which the timer which sets 
up the time amount which confirms the following gesture is started are set up. 
[0023] Moreover, the pointer management table 22 consists of recognition 
[ degree ] gesture items e for recognizing the gesture which should be 
recognized to the gesture actuation name d processed and a degree, and the 



gesture under current processing and the gesture which should be recognized 
next are managed. The block diagram of the management table which manages 
juxtaposition actuation of one example of this invention to drawing 4 is shown. 
The block diagram of the progress processing management table 24 for the 
block diagram of the recognition gesture table 23 where drawing 4 (A) manages 
the operating state of each gesture of the gesture item a, and drawing 4 (B) to 
manage the operating state of processing of the progress processing item c is 
shown. 

[0024] The selection to which the recognition gesture table 23 was set as the 
gesture item a of the gesture actuation management table 21 (1), The effective 
flag f which shows whether (2), cutting (1), (2), the point (1), and (2) can perform 
next It consists of selection object pointers i in which the object made into the 
recognition success flag h which shows whether recognition of the reset flag g 
which shows whether it is necessary to perform recognition processing from an 
initial state, and gesture was successful, and the recognized object is shown. 
[0025] The item which can perform gesture next is set f V), and the item which 
cannot perform gesture is reset for the effective flag f CO'). That is, the gesture 
which should be performed next is shown. Moreover, the reset flag g is set when 
recognition of gesture is not performed fV), and it is reset by the midst by which 
recognition of gesture is performed CO'). 



[0026] Moreover, recognition processing of gesture is performed, the recognition 
success flag h is set when gesture has been recognized ('V), gesture can be 
recognized, and a **** moment and usual are reset f 0*). The selection object 
pointer i registers the pointer for calling the object (icon) recognized by gesture. 
[0027] The progress processing management table 24 consists of unsuccessful 
flags k with which the effective flag j which shows the existence of processing for 
every processing items, such as icon migration set as the progress processing 
item c of the gesture actuation management table 21 and timer starting, and the 
next processing show the existence of activation. The effective flag j is set, when 
gesture is recognized and progress processing is performed (Y), and when 
processing is not performed, it is reset C0^). 

[0028] Moreover, when activation of activation of processing during progress 
processing becomes impossible, the unsuccessful flag k is set (T) and is 
usually held at a reset condition (0 y ). For example, it is set when the following 
gesture is not recognized, even if it carries out fixed time amount progress at the 
time of timer starting. When this unsuccessful flag k is set so that it may mention 
later, the whole system is initialized, and it is constituted so that a system may 
not lapse into a waiting state. 

[0029] First, migration (1) processing is explained. The explanatory view of 
migration processing (1) of one example of this invention of operation is shown 



in drawing 5 . In migration processing (1), it tries to recognize selection (1) 
gesture first. Here, selection (1) gesture is a motion which separates from a 
screen the pen 2 which has touched the object of the icon 31 grade displayed on 
the display 4 as shown in drawing 5 (A) through a tablet 3. In addition, the 
location which the pen 2 is directing is displayed as pen cursor 32 on a screen. A 
screen display of the pen cursor 32 is carried out as an arrow head of void. 
[0030] When this selection (1) gesture has been recognized, the gesture 
actuation name d of the pointer management table 22 is set as migration 
processing (1), and *Z the recognition [ degree ] gesture item e indicates the 
2nd step of the gesture item a of migration processing (1) of the gesture 
actuation management table 21 to be is set up. Moreover, the image 
effectiveness item b of the gesture actuation management table 21 is carried out, 
and as shown in drawing 5 (B), a shadow 33 is attached to an icon 31 as image 
effectiveness. Processing to which an icon 31 is moved according to the location 
of a pen 2 is carried out until the point (1) gesture which is the following gesture 
as progress processing c at this time is recognized. Here, point (1) gesture is 
actuation to which a pen 2 is contacted on a screen. 

[0031] Next, if point (1) gesture is recognized, the shadow 33 attached to the 
icon 31 will be eliminated, and migration of an icon 31 will be suspended. At this 
time, by migration processing (1), as shown in drawing 5 (C), when migration of 



an icon 31 stops, icon 3V is displayed on the original icon location with a broken 
line. Here, as shown in drawing 5 (D), when icon 31 * of the broken line of the 
original location is again contacted with a pen 2 in fixed time amount, icon 3V of 
origin is again displayed on the original icon location as a continuous line, and it 
considers as the configuration which can copy an icon. For this reason, a timer is 
started as progress processing c at the time of point (1) gesture recognition, and 
when the selection (1) gesture which is the following gesture is carried out 
before the timer was turned off, continuous-line-ization of the icon 31 of the 
image effectiveness item b is performed. 

[0032] Next, migration processing (2) is explained. The explanatory view of 
migration processing (2) of one example of this invention of operation is shown 
in drawing 6 . In migration processing (2), as shown in drawing 6 (A), selection of 
an icon 31 is performed by moving so that the perimeter of an icon 31 to which 
you are going to make it move without contacting a pen 2 to a display side by 
selection (2) may be surrounded, and a shadow 33 is attached to the icon 31 
chosen as shown in drawing 6 (B). If a timer is started and the point (1) which is 
the following gesture is recognized in fixed time amount at this time, an icon 31 
will be moved to the location P1 where the icon 31 was directed on the point (1) 
from the original location P0 as shown in drawing 6 (C). 

[0033] Moreover, even if fixed time amount passes, when no gesture is carried 



out, the shadow 33 attached to the icon as image effectiveness is eliminated. 
When an icon 31 is moved by the gesture of the point (1), recognition of the 
gesture of the point (2) is stood by next. Here, operation of the gesture of the 
point (2) eliminates the shadow attached to the icon 31 as image effectiveness 
as shown in drawing 6 (D). 

[0034] Next, deletion is explained. The explanatory view of the deletion of one 
example of this invention of operation is shown in drawing 7 . Deletion is realized 
by two gesture, cutting (1) and (2). First, a cutting plane line 34 is displayed on 
the icon 31 which it is going to delete as image effectiveness as are shown in 
drawing 7 (A) and the cutting (1) gesture which makes the left cross an icon 31 
top from the right shows a pen 2 to drawing 7 (B). At this time, a timer is started 
as progress processing, if the cutting (2) gesture which makes the right cross an 
icon 31 top for a pen 2 from the left in fixed time amount is recognized, as shown 
in drawing 7 (C), in addition to a cutting plane line 34, a cutting plane line 35 will 
be displayed, and an icon 31 will be deleted. 

[0035] Moreover, before fixed time amount passes, by performing the gesture of 
the selection (2) to which a pen 2 is moved so that the perimeter of an icon 31 
shown in drawing 7 (D) may be surrounded, a cutting plane line 34 is eliminated 
and it is restored to the original condition. Furthermore, even if it carries out fixed 
time amount progress, when the gesture of cutting (2) or selection (2) is not 



performed, a cutting plane line 34 is eliminated and it is restored to the original 
condition. 

[0036] As mentioned above, migration processing (1), (2), and deletion are 
performed when CPU5 operates according to the gesture actuation 
management table 21 held at memory 6, the pointer management table 22, the 
recognition gesture table 23, and the progress processing management table 24. 
The operation flow chart of CPU of one example of this invention is shown in 
drawing 8 . 

[0037] CPU5 first initializes the pointer management table 22 stored in memory 6, 
the recognition gesture table 23, and the progress processing management 
table 24 at the time of initiation of operation (step S 1-1). Next, CPU5 acquires 
the information from a tablet 3 by the pen condition recognition section 1 1 , and 
acquires the status information of the pen 2 on a display 4 (step S 1-2). 
[0038] Next, CPU5 recognizes the gesture which the user performed by 
recognition processing later mentioned from the information on the pen 2 
acquired by step S1-2 by the gesture recognition section 12 (step S1-3-S 1-8). If 
gesture is recognized, CPU5 searches the gesture actuation management table 
21, and it will update the contents of the recognition gesture table 23 while it 
carries out progress processing set as the gesture actuation management table 
21 (stepS 1-9, S1-10). 



[0039] Moreover, CPU5 displays the image effectiveness set as the gesture 
actuation management table 21 according to the gesture recognized 
continuously, and returns to step S1-2 (step S 1-13). in addition - when the 
recognized gesture does not exist, and when recognition of gesture goes wrong, 
without it carries out the image effectiveness - step S - gesture is recognized 
again one to 2 return (step S 1-11, S1-12). 

[0040] CPUS repeats the above-mentioned step S1-2-S1-13, and gives the 
image effectiveness beforehand set up to objects, such as an icon displayed on 
the display 4. The operation flow chart of the processing for recognizing 
selection (1) of one example of this invention to drawing 9 is shown. 
[0041] In the processing for recognizing the step S1 selection (1) gesture of -3, 
the effective flag f of the recognition gesture table 23 is detected first (step S 2-1). 
Here, if the effective flag f is reset, since it is in the condition that gesture cannot 
be recognized, recognition processing is ended. Moreover, if the effective flag f 
is set, since it is in the condition that gesture can be recognized, the reset flag g 
of the recognition gesture table 23 is detected next (step S 2-2). If the reset flag 
g is set, since it shows that it is necessary to recognize from an initial state, it is 
judged whether whether the pen's 2 touching the display 4 and a pen 2 touch the 
tablet 3 first (step S 2-3). Since it is not an initial state when the pen 2 touches 
the display 4, processing is ended as it is. Moreover, when the pen 2 does not 



touch a display 4, the reset flag g is reset, in order to recognize a series of 
actuation of a pen 2, the coordinate list which carried out sequential storing of a 
series of coordinates is cleared, and the coordinate data acquired at this time is 
set as a head coordinate (step S2-4-S 2-6). Initialization is completed by the 
above. 

[0042] Next, the acquired coordinate data is added to a coordinate list (step S 
2-7). Here, when the reset flag g is reset by step S2-2, coordinate data is 
continuously added by step S2-7. Here, it is judged for the added coordinate 
data whether it is the head coordinate of a coordinate list (step S 2-8). Step S 
Since it is not in the condition that selection (1) was still chosen when the 
directions location of a pen 2 is a head coordinate in 2-8, processing is ended. 
Moreover, when coordinate data is not a head coordinate in step S2-8, in order 
to judge whether it is selection (1) processing, it is judged whether next it is 
located in a head coordinate neighborhood (step S 2-9). 

[0043] Step S Since it can judge that selection (1) is not chosen when there is no 
coordinate of a pen 2 in the head coordinate neighborhood of a coordinate list 
2-9, the reset flag g of selection (1) is set and processing is ended (step S 2-14). 
Moreover, since selection (1) may be chosen when the coordinate of a pen 2 has 
stopped at the head coordinate neighborhood of a coordinate list step S2-9, it is 
judged whether the pen 2 which is next the gesture of selection (1) touches the 



display 4 (stepS 2-10). 

[0044] Step S If the pen 2 does not touch a display 4 by 2-10, since it cannot 
judge that selection (1) was chosen, processing is ended. Moreover, if the pen 2 
touches the display 4 by step S2-10, it can judge that selection (1) was chosen 
and the icon located in the coordinate which the pen 2 is directing next will be 
searched (step S 2-11). 

[0045] Step S If an icon is in the coordinate which the pen 2 is directing as a 
result of retrieval of 2-11, the recognition success flag h of the recognition 
gesture table 23 will be set as an icon chosen by selection (1) in the icon, and 
the information which identifies the icon chosen as the selection object pointer i 
will be set up (step S 2-12, S2-13). Moreover, if an icon does not exist in the 
coordinate which the pen 2 is directing as a result of retrieval of step S2-11, 
since it cannot judge that selection (1) was carried out, the reset flag g of 
selection (1) is set and processing is ended (step S 2-12, S2-14). 
[0046] The gesture of selection (1) is recognized by the above. The operation 
flow chart of the processing for recognizing selection (2) of one example of this 
invention to drawing 10 is shown. In the processing for recognizing the step S1 
selection (2) gesture of -4, the effective flag f of the recognition gesture table 23 
is detected first (step S 3-1). Here, if the effective flag f is reset, since it is in the 
condition that gesture cannot be recognized, recognition processing is ended. 



Moreover, if the effective flag f is set, since it is in the condition that gesture can 
be recognized, the reset flag g of the recognition gesture table 23 is detected 
next (step S 3-2). If the reset flag g is set, since it shows that it is necessary to 
recognize from an initial state, it is judged whether whether the pen's 2 touching 
the display 4 and a pen 2 touch the tablet 3 first (step S 3-2). Since it is not an 
initial state when the pen 2 touches the display 4, processing is ended as it is. 
Moreover, when the pen 2 does not touch a display 4, the reset flag g is reset, in 
order to recognize a series of actuation of a pen 2, the coordinate list which 
carried out sequential storing of a series of coordinates is cleared, and the 
coordinate data acquired at this time is set as a head coordinate (step S3-4-S 
3-6). Initialization is completed by the above. 

[0047] Next, the acquired coordinate data is added to a coordinate list (step S 
3-7). Here, when the reset flag g is reset by step S3-2, it is judged whether next 
the pen 2 touches the display 4 (step S 3-8). 

[0048] Step S Since it is not the gesture of selection (2) when the pen 2 touches 
the display 4 by 3-8, the reset flag g is set and processing is ended so that it may 
next be in an initial state and may be set up (step S 3-9). Moreover, since it may 
be the gesture of selection (2) when the pen 2 does not touch a display 4 by step 
S3-8, the coordinate which the pen 2 is directing then is added to the coordinate 
list holding the locus of a pen 2 (step S 3-7). 



[0049] Step S If coordinate data is newly added to a coordinate list by 3-7 next, it 
will be judged for the added coordinate data whether it is the head coordinate of 
a coordinate list (step S 3-10). Step S Since it is not in the condition that 
selection (2) was still chosen when the directions location of a pen 2 is a head 
coordinate in 3-10, processing is ended. Moreover, when coordinate data is not 
a head coordinate in step S3-10, in order to judge whether it is selection (2) 
processing, it is judged whether next it is located in a head coordinate 
neighborhood (step S 3-11). 

[0050] or [ that selection (2) is not chosen when there is no coordinate of a pen 2 
in the head coordinate neighborhood of a coordinate list step S3-11 ] -- or the 
gesture of selection (2) - on the way ~ since it can judge that it comes out, 
processing is ended as it is. Moreover, since selection (2) may be chosen when 
the coordinate of a pen 2 exists in the head coordinate neighborhood of a 
coordinate list by step S3-1 1 , the icon contained in the closed region formed of 
the locus of the pen 2 which is next the gesture of selection (2) is searched (step 
S3-12). 

[0051] Step S If an icon is in the closed region formed of the locus of the 
coordinate which the pen 2 directed as a result of retrieval of 3-12, the 
recognition success flag h of the recognition gesture table 23 will be set as an 
icon chosen by selection (2) in the icon, and the information which identifies the 



icon chosen as the selection object pointer i will be set up (step S 3-13, S3-14). 
Moreover, if an icon does not exist in the closed region formed of the locus of the 
coordinate which the pen 2 directed as a result of retrieval of step S3-13, since it 
cannot judge that selection (2) was carried out, the reset flag g of selection (2) is 
set and processing is ended (S3-13, S3-9). 

[0052] The gesture of selection (2) is recognized by the above. The operation 
flow chart of the processing for recognizing cutting (1) of one example of this 
invention to drawing 11 is shown. In the processing for recognizing the step S1 
cutting (1) gesture of -5, the effective flag f of the recognition gesture table 23 is 
detected first (step S4 -1). Here, if the effective flag f is reset, since it is in the 
condition that gesture cannot be recognized, recognition processing is ended. 
Moreover, if the effective flag f is set, since it is in the condition that gesture can 
be recognized, the reset flag g of the recognition gesture table 23 is detected 
next (step S4 -2). If the reset flag g is set, since it shows that it is necessary to 
recognize from an initial state, it is judged whether whether the pen's 2 touching 
the display 4 and a pen 2 touch the tablet 3 first (step S4 -3). Since it is not an 
initial state when the pen 2 touches the display 4, processing is ended as it is. 
Moreover, when the pen 2 does not touch a display 4, the reset flag g is reset, in 
order to recognize a series of actuation of a pen 2, the coordinate list which 
carried out sequential storing of a series of coordinates is cleared, and the 



coordinate data acquired at this time is set as a head coordinate (step S4-4-S 
2-6). Initialization is completed by the above. 

[0053] Next, the acquired coordinate data is added to a coordinate list (step S4 
-7). Here, when the reset flag g is reset by step S4 -2, coordinate data is 
continuously added by step S4 -7. Here, it is judged for the added coordinate 
data whether it is the head coordinate of a coordinate list (step S4 -8). Since it is 
not in the condition that cutting (1) was still chosen when the directions location 
of a pen 2 is a head coordinate in step S4 -8, processing is ended. Moreover, 
when coordinate data is not a head coordinate in step S4 -8, in order to judge 
whether it is cutting (1) processing, it is judged whether the pen 2 which is next 
the gesture of cutting (1) touches the display 4 (step S4 -9). 
[0054] If the pen 2 does not touch a display 4 by step S4 -9, since it cannot judge 
that cutting (1) was chosen, processing is ended. Moreover, if the pen 2 touches 
the display 4 by step S4 -9, it can judge that cutting (1) may be chosen and the 
icon which the locus of a pen 2 crosses on the right from the left from a 
coordinate list next will be searched (step S4 -1 0). 

[0055] If there is an icon which the locus of a pen 2 crosses on the right from the 
left as a result of retrieval of step S4 -10, the recognition success flag h of the 
recognition gesture table 23 will be set as an icon chosen by cutting (1) in the 
icon, and the information which identifies the icon which it is going to cut to the 



selection object pointer i will be set up (step S4 -12). Moreover, if the icon which 
the locus of a pen 2 crosses on the right from the left does not exist as a result of 
retrieval of step S4 -10, since it cannot judge that cutting (1) was carried out, the 
reset flag g of cutting (1) is set and processing is ended (step S4 -13). 
[0056] The gesture of cutting (1) is recognized by the above. The operation flow 
chart of the processing for recognizing cutting (2) of one example of this 
invention to drawing 12 is shown. In the processing for recognizing the step S1 
cutting (2) gesture of -6, the effective flag f of the recognition gesture table 23 is 
detected first (step S 5-1). Here, if the effective flag f is reset, since it is in the 
condition that gesture cannot be recognized, recognition processing is ended. 
Moreover, if the effective flag f is set, since it is in the condition that gesture can 
be recognized, the reset flag g of the recognition gesture table 23 is detected 
next (step S 5-2). If the reset flag g is set, since it shows that it is necessary to 
recognize from an initial state, it is judged whether whether the pen's 2 touching 
the display 4 and a pen 2 touch the tablet 3 first (step S 5-3). Since it is not an 
initial state when the pen 2 touches the display 4, processing is ended as it is. 
Moreover, when the pen 2 does not touch a display 4, the reset flag g is reset, in 
order to recognize a series of actuation of a pen 2, the coordinate list which 
carried out sequential storing of a series of coordinates is cleared, and the 
coordinate data acquired at this time is set as a head coordinate (step S5-4-S 



5-6). Initialization is completed by the above. 

[0057] Next, the acquired coordinate data is added to a coordinate list (step S 
5-7). Here, when the reset flag g is reset by step S5-2, coordinate data is 
continuously added by step S5-7. Here, it is judged for the added coordinate 
data whether it is the head coordinate of a coordinate list (step S 5-8). Since it is 
not in the condition that cutting (2) was still chosen when the directions location 
of a pen 2 is a head coordinate in step S4 -8, processing is ended. Moreover, 
when coordinate data is not a head coordinate in step S5-8, in order to judge 
whether it is cutting (2) processing, it is judged whether the pen 2 which is next 
*■ the gesture of cutting (2) touches the display 4 (step S 5-9). 

[0058] Step S If the pen 2 does not touch a display 4 by 5-9, since it cannot 
judge that cutting (2) was chosen, processing is ended. Moreover, if the pen 2 
touches the display 4 by step S5-9, it can judge that cutting (2) may be chosen 
and the icon which the locus of a pen 2 crosses on the right from the left from a 
coordinate list next will be searched (step S 5-10). 

[0059] Step S If there is an icon which the locus of a pen 2 crosses on the left 
from the right as a result of retrieval of 5-10, the recognition success flag h of the 
recognition gesture table 23 will be set as an icon chosen by cutting (2) in the 
icon, and the information which identifies the icon which it is going to cut to the 
selection object pointer i will be set up (step S 5-12). Moreover, if the icon which 



the locus of a pen 2 crosses on the left from the right does not exist as a result of 
retrieval of step S5-10, since it cannot judge that cutting (2) was carried out, the 
reset flag g of cutting (1) is set and processing is ended (step S 5-13). 
[0060] The gesture of cutting (2) is recognized by the above. The operation flow 
chart of the processing for recognizing the point (1) of one example of this 
invention to drawing 13 is shown. In the processing for recognizing the step S1 
point (1) gesture of -7, the effective flag f of the recognition gesture table 23 is 
detected first (step S 6-1). Here, if the effective flag f is reset, since it is in the 
condition that gesture cannot be recognized, recognition processing is ended. 
Moreover, if the effective flag f is set, since it is in the condition that gesture can 
be recognized, the reset flag g of the recognition gesture table 23 is detected 
next (step S 6-2). If the reset flag g is set, since it shows that it is necessary to 
recognize from an initial state, it is judged whether whether the pen's 2 touching 
the display 4 and a pen 2 touch the tablet 3 first (step S 6-3). When the pen 2 
touches the display 4, since the point (1) cannot be recognized, it ends 
processing as it is. Moreover, when the pen 2 does not touch a display 4, the 
reset flag g is reset and it sets up that recognition processing is performing (step 
S 6-4). 

[0061] In order to judge whether it is point (1) processing, it is judged whether 
the pen 2 which is next the gesture of the point (1) touches the display 4 (step S 



6-5). Step S If the pen 2 does not touch a display 4 by 6-5, since it cannot judge 
that the point (1) was chosen, processing is ended. Moreover, if the pen 2 
touches the display 4 by step S6-5, it can judge that the point (1) may be chosen 
and the icon of the location which the pen 2 is directing will be searched (step S 
6-6). 

[0062] Step S If an icon is in the location which the pen 2 is directing as a result 
of retrieval of 6-6, the recognition success flag h of the recognition gesture table 
23 will be set as an icon chosen by the point (1) in the icon, and the information 
which identifies the icon directed to the selection object pointer i will be set up 
(step S 6-7). Moreover, if an icon does not exist in the location which the pen 2 is 
directing as a result of retrieval of step S6-6, since it cannot judge that the point 
(1) was carried out, the reset flag g of the point (1) is set and processing is 
ended (step S 6-8). 

[0063] The gesture of the point (1) is recognized by the above. The operation 
flow chart of the processing for recognizing the point (2) of one example of this 
invention to drawing 14 is shown. In the processing for recognizing the step S1 
point (2) gesture of -8, the effective flag f of the recognition gesture table 23 is 
detected first (step S 7-1). Here, if the effective flag f is reset, since it is in the 
condition that gesture cannot be recognized, recognition processing is ended. 
Moreover, if the effective flag f is set, since it is in the condition that gesture can 



be recognized, the reset flag g of the recognition gesture table 23 is detected 
next (step S 7-2). If the reset flag g is set, since it shows that it is necessary to 
recognize from an initial state, it is judged whether whether the pen's 2 touching 
the display 4 and a pen 2 touch the tablet 3 first (step S 7-3). When the pen 2 
does not touch a display 4, since the point (2) cannot be recognized, it ends 
processing as it is. Moreover, when the pen 2 touches the display 4, the reset 
flag g is reset and it sets up that recognition processing is performing (step S 
7-4). Initialization is completed by the above. 

[0064] In order to judge whether it is point (2) processing, it is judged whether 
*■ the pen 2 which is next the gesture of the point (2) touches the display 4 (step S 
7-5). Step S If the pen 2 touches the display 4 by 7-5, since it cannot judge that 
the point (2) was chosen, processing is ended. Moreover, if the pen 2 does not 
touch a display 4 by step S7-5, it can judge that the point (2) may have been 
chosen and the icon of the location which the pen 2 is directing is searched (step 
S 7-6). 

[0065] Step S If an icon is in the location which the pen 2 is directing as a result 
of retrieval of 7-6, the recognition success flag h of the recognition gesture table 
23 will be set as an icon chosen by the point (2) in the icon, and the information 
which identifies the icon directed to the selection object pointer i will be set up 
(step S 7-7). Moreover, if an icon does not exist in the location which the pen 2 is 



directing as a result of retrieval of step S6-6, since it cannot judge that the point 
(2) was carried out, the reset flag g of the point (2) is set and processing is 
ended (step S 7-8). 

[0066] The gesture of the point (2) is recognized by the above. As shown in 
drawing 8 , sequential execution of the recognition processing as which 
recognition processing of step S1-3-S1-8 explained layer structure to drawing 9 - 
drawing 14 while acquiring the coordinate of nothing and a pen 2 and the contact 
condition with a display 4 is carried out, and CPU5 displays the image 
effectiveness according to a recognition result. Thus, according to this example, 
it can recognize now intuitively that considered as the display when it was going 
to move an icon, as if the shadow was attached to the icon as image 
effectiveness and the icon lost touch with a screen by choosing the icon which is 
going to move by selection (1) or (2), and the icon was chosen. Moreover, when 
deleting an icon, by writing x to the icon which it is going to delete with a pen 2, a 
cutting plane line is displayed on an icon and that an icon is deleted after this can 
recognize intuitively. An icon will be restored if the gesture which writes O with a 
pen 2 on the way is performed at this time. Thus, deletion and migration of an 
icon can be performed intuitively and a user-friendly input/output interface can 
be realized. 

[0067] In addition, in this example, although the display control at the time of 



migration processing and deletion was explained, if it does not restrict to this and 
the display control at the time of other processings is set as the gesture 
management table 21, an intuitive display as well as the migration processing 
and deletion which were mentioned above can be performed. Moreover, in this 
example, although explanation about migration and deletion of an icon is given, 
it does not restrict to an icon and can respond similarly to other graphic form and 
alphabetic characters. 
[0068] 

[Effect of the Invention] Since the sequential grasp of the processing situation of 
an object can be carried out by making a sequential change according to 
advance of processing of the display of an object of an object according to claim 
1 of this invention, it has the features of being able to grasp a processing 
situation intuitively. 

[0069] By attaching a shadow to the object chosen when an object was chosen 
according to claim 2, the selected object loses touch with a screen, is displayed, 
and has the features of being able to grasp the selected object intuitively. Since 
according to claim 3 it is displayed as if the object was cut on the screen by 
displaying a cutting plane line on an object, when deleting an object, it has the 
features of being able to grasp intuitively the processing which deletes an object. 
[0070] Since according to claim 4 an object is deleted by performing actuation 



which cuts an object with a directions means when deleting an object, it has the 
features of being able to perform processing which deletes an object in intuitive 
actuation. When moving an object, after choosing an object with a directions 
means, in order according to claim 5 to attach a shadow to an object and to 
move said object according to migration of a directions means by making a 
directions means estrange from a screen, such a display to which raise an object 
with a directions means and it is made to move is performed, therefore it has the 
features of being able to grasp migration of an object intuitively. 
[0071] Since an object can be chosen with feeling which surrounds the object 
chosen by considering actuation which chooses an object as the actuation which 
surrounds an object with a directions means with a circle according to claim 6, it 
has the features of being able to choose an object intuitively. 
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[Brief Description of the Drawings] 

[Drawing 1] It is the block block diagram of one example of this invention. 
[Drawing 2] It is the block block diagram of the important section of one example 
of this invention. 

[Drawing 3] It is the block diagram of the management table which manages 
gesture actuation of one example of this invention. 

[Drawing 4] It is the block diagram of the management table which manages 
juxtaposition actuation of one example of this invention. 
[Drawing 5] It is the explanatory view of migration processing (1) of one example 
of this invention of operation. 

[Drawing 6] It is the explanatory view of migration processing (2) of one example 
of this invention of operation. 

[Drawing 7] It is the explanatory view of the deletion of one example of this 
invention of operation. 

[Drawing 8] It is the operation flow chart of CPU of one example of this invention. 
[Drawing 9] It is the operation flow chart of the processing for recognizing 
selection (1) of one example of this invention. 

[Drawing 10] It is the operation flow chart of the processing for recognizing 
selection (2) of one example of this invention. 



[Drawing 11] It is the operation flow chart of the processing for recognizing 
cutting (1 ) of one example of this invention. 

[Drawing 12] It is the operation flow chart of the processing for recognizing 
cutting (2) of one example of this invention. 

[Drawing 13] It is the operation flow chart of the processing for recognizing the 
point (1) of one example of this invention. 

[Drawing 14] It is the operation flow chart of the processing for recognizing the 
point (2) of one example of this invention. 
[Description of Notations] 

1 Computer System 

2 Pen 

3 Tablet 

4 Display 

5 CPU 

6 Memory 

11 Pen Condition Recognition Section 

12 Gesture Recognition Section 

13 Object Selection Section 

14 Visual Effect Section 

21 Gesture Actuation Management Table 



22 Pointer Management Table 

23 Recognition Gesture Table 

24 Progress Processing Management Table 

31 Icon 

32 Pen Cursor 

33 Shadow 

34 35 Cutting plane line 



